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Abstract 

We propose a new quantifier elimination algorithm for the theory of 
linear real arithmetic. This algorithm uses as subroutines satisfiability 
modulo this theory and polyhedral projection; there are good algorithms 
and implementations for both of these. The quantifier elimination al- 
gorithm presented in the paper is compared, on examples arising from 
program analysis problems and on random examples, to several other im- 
plementations, all of which cannot solve some of the examples that our 
algorithm solves easily. 

1 Introduction 

Consider a logic formula F, possibly with quantifiers, whose variables lay within 
a certain set S and whose atomic predicates are relations over S. The models of 
this formula are assignments of values in S for the free variables of F such that 
F evaluates to "true". Quantifier elimination is the act of providing another 
formula F' , without quantifiers, such that F and F' are equivalent, that is, have 
exactly the same models. For instance, \fx (x > y =4> x > 3) is equivalent to 
quantifier- free y > 3. 

If F has no free variables, then F' is a ground (quantifier-free, variable-free) 
formula. In most practical cases such formulas can be easily decided to be true 
or false; quantifier elimination thus provides a decision procedure for quantified 
formulas. 

In this paper, we only consider relations of the form L(x, y, z, . . . ) > where 
L is a linear affine expression (an arithmetic expression where multiplication 
is allowed only by a constant factor), interpreted over the real numbers (or, 
equivalently, over the rationals) . We can thus deal with any formula over linear 
equalities or inequalities. Our algorithm transforms any formula of the form 
3xi, . . . ,x n F, where F has no quantifiers, into a quantifier-free formula F' in 
disjunctive normal form. Nested quantifiers are dealt with by syntactic induc- 
tion: in order to eliminate quantifiers from 3x F or Vie F, where F may contain 
quantifiers, one first eliminates quantifiers from F. Universal quantifiers are 
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converted to existential ones (Vxi, . . . , x n F = Sxi, . . . , x n ~<F), yet our algo- 
rithm generally avoids the combinatorial explosion over negations that hinders 
some other methods. 

Our method can be understood as an improvement over the approach of 
converting to DNF through ALL-SAT and performing projection; we compared 
both approaches experimentally (see § I5.2p . We compared our implementa- 
tion with commercial and noncommercial quantifier elimination procedures over 
some examples arising from practical program analysis cases, as well as ran- 
dom problems, and ours was the only one capable of processing them without 
exhausting memory or time, or failing altogether due to the impossibility of 
handling large coefficients. 

2 The Algorithm 

We first describe the datatypes on which our algorithm operates, then the off- 
the-shelf subroutines that it uses, then the algorithm and its correctness proof, 
then possible alterations. 

2.1 Generalities 

We operate on unquantified formulas built using A, V, =$■, -i or other logical 
connectives such as exclusive-or (the exact set of connectives allowed depends 
on the satisfiability tester being used, see below; in this paper we shall only use 
A, V and ->), and on quantified formulas built with the same connectives and the 
existential (3) and universal (V) quantifiers. It is possible to quantify not only 
on a single variable but also on a set of variables, represented as a vector v. The 
atoms are linear inequalities, that is, formulas of the form c+c x x+c y y+c z z ■ ■ ■ > 
where c € Q is the constant coefficient and c v € Q is the coefficient associated 
with variable v. It is trivially possible to represent equalities or strict inequalities 
using this formula language. The models of a formula F are assignments a of 
rational numbers to the free variables of F such that a satisfies F (written 
a \= F). F is said to be satisfiable if a model exists for it. If F has no free 
variables, then F is said to be true if F is satisfiable, false otherwise. Two 
formulas A and B are said to be equivalent, noted A = B, if they have the same 
models. Formula A is said to imply formula B, noted A ^ B, if any model of 
A is a model of B. 

Consider a quantifier-free formula F, whose atomic predicates are linear 
inequalities, and variables x\, . . . ,x n . We wish to obtain a quantifier- free for- 
mula F' equivalent to 3xi,...,x n F. Let us temporarily forget about effi- 
ciency in order to convince ourselves quickly that quantifier elimination is pos- 
sible. F can be put into disjunctive normal form (DNF) C\ V • • • V C m (by 
recursive application of distributivity), and 3xi, ...,x n F is thus equivalent to 
(3xi, . . . , x n C\) V ■ • • V (3xi, . . . , x n C m ). Various methods exist for finding a 
conjunction C- equivalent to 3x±, . . . , x n Ci, among which we can cite Fourier- 
Motzkin elimination (see § I5.1[) . We therefore obtain F' in DNF. For a universal 
quantifier, through De Morgan's laws, we obtain a formula in conjunctive normal 
form (CNF). 

Such a naive algorithm suffers from an obvious inefficiency, particularly if 
applied recursively to formulas with alternating quantifiers. A first and obvious 
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step is to replace DNF conversion through distributivity by modern techniques 
(model enumeration using satisfiability modulo theory). We show in this paper 
than one can do better by interleaving the projection and the model numeration 
processes. 

2.2 Building blocks 

If one has propositional formulas with a large number of variables, one never 
converts formulas naively from CNF to DNF, but one uses techniques such as 
propositional satisfiability (SAT) solving. Even though SAT is NP-complete, 
there now exist algorithms and implementations that can deal efficiently with 
many large problems arising from program verification. In our case, we apply 
SAT modulo the theory of linear real inequalities (SMT), a problem for which 
there also exist algorithms, implementations, standard benchmarks and even a 
competition. Like SAT, SAT modulo linear inequalities is NP-complete. A SMT 
solver takes as an input a formula F where the literals are linear equalities or 
inequalities, and answers either "not satisfiable", or a model of F, assigning a 
rational number to each variable in F. We assume we have such an algorithm 
Smt at our disposal as a building block 

Another needed building block is quantifier elimination over conjunctions, 
named Project(C, v): given a conjunction C of linear inequalities over vari- 
ables {?=«!,..., vn, obtain a conjunction C equivalent to 3v\, . . . , v n C. For 
efficiency reasons, it is better if C" is minimal (no conjunct can be removed 
without adding more models), or at least "small". Fourier- Motzkin elimina- 
tion is a simple algorithm, yet, when it eliminates a single variable, the output 
conjunction can have a quadratic number of conjuncts compared to the in- 
put conjunction, thus a pass of simplification is needed for practical efficiency; 
various algorithms have been proposed in that respect [H] . For our implementa- 
tions, we used "black box" libraries implementing geometrical transformations, 
in particular polyhedron projection: C defines a convex polyhedror0 in Q> N , and 
finding C amounts to computing the inequalities defining the projection of this 
polyhedron into Q N ~ n . 

3 Quantifier Elimination Algorithm 

We shall first describe subroutines GeneralizeI and Generalize2, then the 
main algorithm ExistElim. 

3.1 Generalized Models 

Consider a satisfiable quantifier-free formula F. We suppose we have at our 
disposal a SMT-solving algorithm that will output a model m \= F. We wish 
to obtain instead a generalized model: a conjunction C such that C ==> F. 

1 A good bibliography on convex polyhedra and the associated algorithms can be found 
in the documentation of the Parma Polyhedra Library, [I] By convex polyhedron, we mean, 
in a finite-dimension affine linear real space, an intersection of a finite number of half-spaces 
each delimited by a linear inequality, that is, the set of solutions of a finite system of linear 
inequalities. In particular, such a polyhedron can be unbounded. In the rest of the paper, the 
words "polyhedron" must be understood to mean "convex polyhedron" with that definition. 
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Algorithm 1 Generalize!. (a, F): Generalize a model a of a formula F to a 
conjunction 
Require: a \= F 
M «- true 

for all P e AtomicPredicates(F) do 
if a \= P then 
M <- M AP 
else 

M <- M A ->P 
end if 
end for 
Ensure: M ^ F 



Algorithm 2 Generalize2(G, M): Remove useless constraints from conjunc- 

tion M so that GAM = false 

Require: G A M is not satisfiable 
for all c conjunct in M do 

if (G \ {c}) A M is not satisfiable (call Smt) then 

remove c from M 
end if 
end for 

Ensure: G A M is not satisfiable 



Ideally, we would like C to have as few conjuncts as possible. We shall now see 
algorithms in order to obtain such generalized models. 

The truth value of F on an assignment a of its variables only depends 
on the truth value of the atomic predicates of F over a. Let us note Np — 
| AtomicPredicates (F) I , where | X | denotes the cardinality of the set X . These 
truth assignments therefore define at most 2 Np equivalence classes over the val- 
uations of the variables appearing in F. There can be fewer than 2 Np equiva- 
lence classes, because some truth assignments can be contradictory (for instance, 
x > 1 assigned to true and x > assigned to false). One can immediately gener- 
alize a model of a formula to its equivalence class, which motivates our algorithm 
GeneralizeI. Its output is a conjunction of literals from F. 

This conjunction may itself be insufficiently general. Consider the formula 
F = (x > A y > 0) V (-nr. > A y > 0). x h-> 0,y i-> is a model of F. 
GeneralizeI will output the conjunction x > OAy > 0. Yet, the first conjunct 
could be safely removed. Generalize2(-i(F V O), M) will remove unnecessary 
conjuncts from M while preserving the property that M ^ F V O. Figure [3] 
illustrates why it is better to generalize the conjunctions. 

The problem of obtaining a minimal (or at least, "reasonably small") in- 
consistent subset out of an inconsistent conjunction has already been studied. 
In DPLL(T) algorithms [8 for SMT-solving, the problem is to find out, given 
a consistent conjunction of literals L\ A ■ ■ ■ A L n and a new literal V , whether 
L\ A - ■ ■ AL n L', L\ A - ■ ■ AL n =>■ -uV, or neither; and if one of the implications 
holds, produce a minimal explanation why it holds, that is, a subset , . . . , Li m 
of the Li such that A • • • A L im => L' (respectively, => ->L'). Since this de- 
cision and explanation procedure is called often, it should be fast and much 
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projection of A 



Figure 1: Subsumption of one generalized model by another 

effort has been devoted in that respect by implementors of SMT-solvers (e.g. 
[T5] for congruence theories). It is however not straightforward to use such ex- 
planation procedures for our purposes, since we do not consider conjunctions of 
literals only: when algorithm ExistElim invokes Generalize2(-iF, Mi), 
is in general a complex formula, not a literal. 

We therefore present here a straightforward inconsistent set minimization 
algorithm similar to the one found in [HI §6]. Generalize2(G, M), where M is 
a conjunction such that G A M is unsatisfiable, works as follows: 

• It attempts removing the first conjunct from M (thus relaxing the M 
constraint). If G A M stays unsatisfiable, the conjunct is removed. If it 
becomes satisfiable, then the conjunct is necessary and is kept. 

• The process is continued with the following conjuncts. 

Unsurprisingly, the results of this process depend on the order of the con- 
juncts inside the conjunction M . Some orders may perform better than others; 
the resulting set of conjuncts is minimal with respect to inclusion, but not nec- 
essarily with respect to cardinality. H 

3.2 Main Algorithm 



2 This is the case even if we consider a purely propositional case. As an example, consider 
F = AV (B A C). M = A A B A C ^ F, otherwise said M A -<F is not satisfiable. If one 
first relaxes the constraint A, one gets the conjunction B A C, which still implies F\ this 
conjunction has two propositional models (A A B A C and —>A ABA C). Yet, one could have 
chosen to relax B and obtain A A C, and then to relax C and obtain A (which still implies 
F); this formula has four propositional models. 
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Figure 2: The gray area is the set of points matched by formula F — y > 
-1 V (y > -2 A x > -1 A x < 1). Point O = (0,0) is found as a model. This 
model is first generalized to y > — 1 Ay > — 2 A x > — lAi < 1 according to 
its valuations on the atomic boolean formulas. Depending on whether one first 
tries to relax x > — 1 or y > — 1, one gets either a half plane (one conjunct) or 
a vertical band (three conjuncts); the former is "simpler" than the second. The 
simplicity of the formula output by Generalize2 thus depends on the ordering 
of the input conjuncts. 



Algorithm 3 ExistElim: Existential quantifier elimination 
H ^F 
O <- false 

while H is satisfiablc (call Smt) do {(3v F) = (O V 3v H) and H A O = false 
and O does not mention variables from v} 

a <— a model of H {a \= H} 

Mi 4- Generalize1(F, a) {Mi ee> F} 

M 2 <— Generalize2(-i_F, Mi) {^(M 2 AG)} 

7T <— Project(M 2 , v) {tt = 3vM 2 } 

H < — HA — i7r 
end while 
Ensure: O = 3v F 



The main algorithm is ExistElim^, v) which computes a DNF formula 
equivalent to 3v F. v is a vector of variables, v can be empty, and then the al- 
gorithm simply computes a "simple" DNF form for F. The algorithm computes 
generalized models of F and projects them one by one, until exhaustion. It 
maintains three formulas H and O. O is a DNF formula containing the projec- 
tions of the models processed so far. H contains the models yet to be processed; 
it is initially equal to F. For each generalized model M, its projection tt is 
added to O and removed from H. ExistElim can thus be understood as an 
ALL-SAT implementation coupled with a projection, where the projection is 
performed inside the loop so as to simplify the problem (as opposed to waiting 
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Figure 3: A is the first generalized model selected. If Go = ~^F, the initial 

dcf 

value of G, is replaced at the next iteration by G\ — -<F A ^7r where tto is 
the projection of A, then it is possible to generate a single generalized model 
encompassing both B and C (for instance x > — lAy>0Ay<2. If G stays 
constant, then the x > I constraint defining the left edge of C cannot be relaxed. 

for all models to be output and projecting them). 

The partial correctness of the algorithm ensues from the loop condition and 
the following loop invariants: (3v F) = O V (3v H), H ^ F and O does not 
mention variables from v. 

Given a formula <j>, we denote by W{<f) the number of equivalence classes 
induced by the atomic predicates of F with nonempty intersection with the 
models of (j>. Termination is ensured because W{H) decreases by at least one 
at each iteration: M\ defines exactly one equivalence class, M2 defines a union 
of equivalence classes which includes the one defined by M\ , and the models of 
n include those of M 2 thus also at least one equivalence class. The number of 
iterations is thus at most 2 Nf . Note that Generalize2 is needed neither for 
correctness nor for termination, but only for efficiency: otherwise, the number 
of iterations would always be the number of equivalence classes, which can be 
huge. 

4 Possible Changes and Extensions 

We investigated two variations of the same algorithm, both of which perform 
significantly worse. In addition, we extended the algorithm to quantifier elimi- 
nation modulo a user-specified theory. 
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4.1 ALL-SAT then project (Modi) 

The algorithm would still be correct if M was removed from H instead of tt. 
It then becomes equivalent to performing ALL-SAT (obtaining all satisfying 
assignments) then projection. On the one hand, with this modified algorithm, 
the set of atomic formulas of H would stay included in that of F throughout the 
iterations, while this set can grow larger with the original algorithm since the 
set of atomic formulas of the projection of F can be much larger than the set 
of atomic formulas in F (see ^5.1|) . On the other hand, the original algorithm 
may need fewer iterations because 7r may subsume several generalized models, 
as shown by Fig. [T] : A is the first generalized model being generated, and its 
projection subsumes B; thus, the original algorithm will not have to generate B, 
while the modified algorithm will generate B. Our experiments f ^5.2p showed 
that the unmodified algorithm often performs much better in practice than this 
approach. 

4.2 Removals from Negated Set (Mod2) 



Algorithm 4 ExiSTELiM(Mod2): Existential quantifier elimination 
H <- F 

G «— -^F 
O <- false 

while H is satisfiable (call Smt) do {(3v F) = (O V 3v H) and G = ->(F V O) and 
H A O = false and O does not mention variables from v} 

a <— a model of H {a \= H} 

Mi <- Generalize1(F, a) {Mi ^ F} 

M 2 <- Generalize2(G,Mi) {-.(M 2 A G)} 

7r «— PROJECT(M 2 ,t7) {n = 3v M 2 } 

O <- O Vtt 

H <- H A ^tt 

G ^ GA — i7T 

end while 
Ensure: O = 3v F 



The algorithm given previously was not the first we experimented; we had 
originally a slightly more complicated one, given as ExiSTELiM(Mod2), which 
we wrongly thought would be more efficient. Instead of using -^F to check 
for inappropriate generalizations, we used a formula G initially equal to ->F, 
and then progressively altered. The termination proof stays the same, while 
correctness relies on the additional invariant G = -*(F V O). ExistElim can be 
thought of as identical to ExiSTELiM(Mod2) except that G stays constant. 

We thought this scheme allowed more generalization of models than the 
algorithm we gave earlier in the article, as shown by Fig. [3J ExistElim tries 
to generalize M to a conjunction that implies F, but in fact this is too strict a 
condition to succeed, whereas ExistElim (Mod2) succeeds in generalizing F to 
a conjunction that implies F V O. If at least one variable is projected out, and 
F actually depends on that variable, then the models of F are strictly included 
in those of the final value of O, which is equivalent to 3v F. 

Experiments f q5.2[) however showed that this "more clever" algorithm is 
slower by approximately a factor of two, because adding extra assertions to 



8 



G is costly for the SMT-solver. 
4.3 Extra Modulo Theory 

The algorithm can be easily extended to quantifier elimination modulo an as- 
sumption T on the free variables of F. All definitions stay the same except 
that e> is replaced by =$ T , defined as P ^>t Q = l (P A T) =^ (Q A T) and 

= is replaced by = T , defined as (P = T Q) = f (P A T = Q A T). ExistElim 
is modified by replacing the initialization of G and H by A T and FAT 
respectively. Intuitively, T defines a universe of validity such that values outside 
of the models T are irrelevant to the problem being studied. 

5 Comparison with Other Algorithms 

The "classical" algorithm for quantifier elimination over linear inequalities is 
Ferrante and Rackoff's [7]. Another algorithm based on similar ideas, but with 
better performance, was proposed by Loos and Weispfenning [10 . We shall 
therefore compare our method to these algorithms, both theoretically and ex- 
perimentally We also compared our algorithm with other available packages 
using other quantifier elimination techniques. 

5.1 Complexity bounds 



Benchmark 


r. lim. R 


r. lim. float 


prsb23 


blowup5 


Mjollnir 


1.4 


17 


0.06 


negligible 


Mjollnir (modi) 


1.6 


77". 


0.06 


negligible 


Mjollnir (mod2) 


1.5 


34 


0.07 


negligible 


Mjollnir Loos-Weispferming 


o-o-m 


o-o-m 


o-o-m 


negligible 


Proof-of-concept 


n/a 


823 


n/a 


n/a 


Mjollnir Ferrante-Rackoff 


o-o-m 


o-o-m 


o-o-m 


negligible 


Proof-of-concept 


n/a 


823 


n/a 


n/a 


Lira 


o-o-m 


o-o-m 


8.1 


0.6 


REDLOG rlqe 


182 


o-o-m 


1.4 


negligible 


REDLOG rlqe+rldnf 


o-o-m 


o-o-m 


n/a 


n/a 


MATHEMATICA Reduce 


(> 12000) 


o-o-m 


(> 780) 


7.36 



"Memory consumption grows to 1.1 GiB. 



Table 1: Timings (in seconds, on an AMD Turion TL-58 64-bit Linux system) 
for eliminating quantifiers from our benchmarks. The first line is the algorithm 
described in this paper, the two following linear variants from §4j then other 
packages. Reduce has rlqe (quantifier elimination) and rlqe+rldnf (same, 
followed by conversion to DNF). (> t) means that the computation was killed 
after t seconds because it was running too long. The prsb23 and following are 
decision problems, the output is true or false, thus DNF form does not matter. 
Out-of-memory is noted "o-o-m". 

We consider in this section that inequalities are written using integer cocffi- 
cients in binary notation. We shall prove that a complexity bound 2" where 
n is the number of atomic formulas and q is the number of quantifiers to be 
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eliminated. This yields an overall complexity of 2 2 where \F\ is the size of 
the formula. 

Let us consider a conjunction of inequalities taken from a set of n inequalities. 
The Fourier-Motzkin algorithm [HE] eliminates variable x from this conjunction 
as follows. It first partitions these inequalities into those where x does not 
appear, which are retained verbatim, and those where x appears positively (E + ) 
and negatively ) . From each couple of inequalities (e+ , e_ ) in E + x £L , an 
inequality where x does not appear is obtained by cancellation between e + and 
e_. The size in bits of the coefficients in the output inequalities can be at most 
2,s + 1 where s is the maximal size of the input coefficients. 

The inequalities output therefore belong to a set of size asymptotically at 
most n 2 /4 (the worst-case occurs when the inequalities split evenly between 
those in which x appears positively and those where it appears negatively). 
The output conjunction is in general too large: many inequalities in it are 
superfluous; yet it is guaranteed to include all inequalities defining the facets of 
the projection of the polyhedron. 

Consider a formula F written with inequalities A\, . . . , A n as atomic for- 
mulas, with maximal coefficient size s. Our algorithm eliminates the quantifier 
from 3x F and outputs a DNF formula F' built with inequalities found in the 
output of the Fourier-Motzkin algorithm operating on the set Ai,. . . ,A n and 
variable x. It follows that F' is built from at most, asymptotically, n 2 /4 in- 
equalities as atomic formulas. The running time for this quantifier elimination 
comes from: 

• The SMT solving passes. There are at most 2™ branches to explore in total. 
For each branch, SMT has to test whether the solution set of a conjunction 
of polynomial inequalities is empty or not, which is a particular case of 
linear programming, with polynomial complexity. The overall SMT cost 
is therefore bounded by 0(2 n .P(n)) for some polynomial P; 

• The projections, with complexity 0(n 2 .s), applied to each of at most 2 n 
polyhedra. 

This gives an overall complexity of 0(2 cn ) where c is a constant. 

Consider now a succession of quantifier eliminations (with or without alter- 
nations). We now have F consisting of a sequence of quantifiers followed by a 
quantifier- free formula built out of atomic formulas A\,... ,A n . Our algorithm 
performs eliminations in sequence, starting from the rightmost quantifier. 

Let us note A^ the set of atomic formulas that can be obtained after k 
eliminations; A^ = {A x , . . . , A n }. Clearly, \A^\ < \A^\ 2 " asymptotically, 
since at each iteration the size of the set of atomic formulas can at most get 
squared by Fourier-Motzkin elimination. The size of the coefficients grows at 
most as s.2 k . This yields the promised bound. 

It is possible that the bound \A^\ < |^4^°^| 2 , obtained by observation 
of the Fourier-Motzkin algorithm, is too pessimistic. The literature does not 
show examples of such doubly exponential blowups, while polyhedra with single 
exponential blowups can be constructed. 

The "classical" algorithm for quantifier elimination over real or rational arith- 
metic is Ferrante and Rackoff's method [7] [11 §7.3] [TH §4.2]. A related algorithm 
was proposed by Loos and Weispfenning [T0JP31 §4-4]. Both these algorithms 
are based on the idea that an existentially quantified formula 3a; F(x) with free 
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variables y,z, . . . can be replaced by F(x\) V • • • V F(x m ) where x\, . . . , x m are 
expressed as functions of y,z, ... . In the case of Ferrante and Rackoff, m is 
quadratic in the worst case in the length of the formula, while for Loos and 
Weispfenning it is linear. In both cases, the overall complexity bound is 2 2 . 

The weakness of both algorithms is that they never simplify formulas. This 
may explain that while their theoretical bounds are better than ours, our algo- 
rithm is in practice more efficient, as shown in the next subsection. 

One could at first assume that the complexity bounds for our algorithm are 
asymptotically worse than Ferrante and Rackoff's. Our algorithm, however, 
outputs results in CNF or DNF form, while Ferrante and Rackoff's algorithm 
does not. If we add a step of transformation to CNF or DNF to their algorithm, 
then we also obtain a triple exponential bound. 

5.2 Practical results 





depth 14 


depth 15 


depth 16 




Solved 


Avg 


O-o-m 


Solved 


Avg 


O-o-m 


Solved 


Avg 


O-o-m 


MJOLLNIR 


100 


1.6 





94 


9.8 





73 


35.3 





Mjollnir (modi) 


94 


8.2 


3 


80 


27.3 


7 


39 


67.1 


25 


Mjollnir (mod2) 


100 


3.8 





91 


13.9 





65 


39.2 





Mjollnir Loos-W. 


93 


1.77 


4 


90 


6.42 


5 


62 


17.65 


27 


Proof-of-concept 


94 


1.4 





86 


2.2 





55 


17.7 





Mjollnir Ferrante-R. 


51 


18.2 


41 


23 


23.2 


65 


3 


7.3 


85 


Proof-of-concept 


94 


1.4 





86 


2.2 





55 


17.7 





Lira 


14 


102.4 


83 


3 


77.8 


94 


1 


8 


95 


REDLOG (rlqe) 


92 


13.7 





53 


27.4 





27 


33.5 





Mathematica 


6 


30.2 





1 


255.7 





1 


19.1 






Table 2: Benchmarks on 3 x 100 random instances generated using randprsb, 
with formula depths n respectively 14, 15 and 16 (obtained byrandprsb 7- 
10 10 n i) where i ranges in [0, 99]). The table shows the number of instances 
solved within the timeout period out of the proposed 100, the average time spent 
per solved instance, and the number of instances resulting in out-of-memory. 

We benchmarked several variants of our method against other algorithms: 

Mjollnir is the algorithm described in implemented on top of SMT solver 
and the NewPolka polyhedron package from APROr\@, or option- 
ally the Parma Polyhedra Library (PPLO). Profiling showed that most of 
the time is spent in the SMT solver, so performance differences between 
NewPolka and PPL are negligible. 

Proof-of-concept is an early version of the same algorithm, implemented on 
top of a rudimentary SMT solver and the PPL. The SMT algorithm used 
is simple and lazy: the SMT problem is turned into SAT by replacing 
each atomic inequality by a propositional variable, and the SAT problem 
is input into Minisat. A full SAT solution is obtained, then tested for 
emptiness by solving a linear programming problem: finding a vector of 

3 http : //yices . csl . sr i . com7"| 

4 http : //apron. cri . ensmp . f r/library/ 
5 http : //www. cs .unipr . it/ppl/ 
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coefficients suitable as a contradiction witness for Farkas' lemma. If a 
witness is found, it yields a contradictory conjunction, whose negation is 
added to the SAT problem and SAT is restarted. 

Mjollnir (modi) is the ALL-SAT then projection algorithm from i j4.ll It is 
invoked by option -no-block-pro jected-model. 

Mjollnir (mod2) is the algorithm from §4.21 it is invoked by option -add- 
blocking-to-g. 

Mjollnir Ferrante-Rackoff implements [7][H §^3]. 

Mjollnir Loos-Weispfenning implements [TO]. 

Lir is based on Biichi automata and handles both Presburger arithmetic (in- 
teger linear inequalities) and rational linear inequalities. 

Mathematical is a general-purpose symbolic algebra package. Its Reduce 
fonction appears to implement CAD [5], an algorithm suitable for non- 
linear inequalities interpreted in the theory of real closed fields, though it 
is difficult to know what exactly is implemented because this program is 
closed source. 

Redlogjl is a symbolic formula package implemented on top of the computer 
algebra system Reduce 3.80 Redlog implements various algorithms due 
to Volker Wcispfcnning and his group pTj . 

Table [TJ compares these various implementations on a few benchmark exam- 
ples coming from two sources: 

1. Examples produced from problems of program analysis following our 
method for the parametric computation of least invariants. |12j To summa- 
rize, each formula expresses the fact that a set of program states (such as a 
product of intervals for the numerical variables) is the least invariant of a 
program, or the strongest postcondition if there is no fixed point involved. 
Most of the examples, being extracted by hand from simple subprograms, 
were easily solved and thus did not constitute good benchmarks, but one 
of them, defining the least invariant of a rate limiter, proved to be tougher 
to solve, and we selected it as a benchmark. We have two versions of this 
example: the first for a rate limiter operating over real numbers ("r. lim 
K") the second over floating-point numbers, abstracted using real numbers 
("r. lim float"), and considerably tougher to process than the real example. 

2. Examples procured from the Lira designers (prsb23 and blowup5). 

Memory consumption stayed modest for all examples (< 15 MiB), except for 
r. lim float. Profiling showed that most of the time is spent in the SMT-solver 
and only a few percents in the projection algorithm. The fact that the proof-of- 
concept implementation, with a very naive SMT-solver, performs decently on an 

e http : //lira.gf orge . avacs . org/ 





'http : //www. wolf ram. com/ 



^http : //www. algebra. fim.uni-passau. de/ "redlog/ 



6 http : //www . uni-koeln . de/REDUCE/ 

rr~ ; ^ — — T. 1. . . . 



Available from http : //www-verimag. imag . f r/~monniaux/download/linear_qe_benchmarks . zip 
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example where other algorithms exhaust memory shows that the performance 
of our algorithm cannot be solely explained by the good quality of Yices. 

Table [2] compares the various al gor ithms on random examples. We then 
used the LIRA team's randprsb tooO to generate 100 random instances, by 
changing the seed of the random number generator from to 99, for each of 
three values (14, 15, 16) of the depth parameter, which measures complexity^ 
The programs were then tested with both a 1.8 GiB memory limit and a timeout 
of five minutes. It is clear from Tab.[5]that Mjollnir -no-add-blocking-to-g 
is the most efficient of the tested tools. 



6 Conclusion and Future Work 

We have proposed a new quantifier elimination algorithm for the theory of linear 
inequalities over the real or rational numbers, and investigated possible variants. 
Our motivation was the practical application of a recent result of ours on pro- 
gram analysis, stating that formulas for computing the least invariants of certain 
kinds of systems can be obtained through quantifier elimination [T2"] . 

This algorithm is efficient on examples obtained from this program analy- 
sis technique, as well as other examples, whereas earlier published algorithms, 
as well as several commercial packages, all exhaust time or memory resources. 
Our algorithm leverages the recent progresses on satisfiability modulo theory 
solvers (SMT) and, contrary to older algorithms, performs on-the-fly simplifi- 
cations of formulas that keep formula sizes manageable. Our algorithm also 
performs better than a straight application of SMT solvers (ALL-SAT followed 
by projection). 

Our algorithm is described for rational or real linear arithmetic, but it can 
be extended to any theory for which there is an efficient satisfiability testing 
algorithm for unquantificd formulas and a reasonably efficient projection al- 
gorithm for conjunctions. Among extensions that could be interesting from a 
practical point of view would be on the one hand the nonlinear case for real 
arithmetic (polynomials), and on the other hand the mixed integer / real prob- 
lems. Of course, nonlinear integer arithmetic cannot be considered, since Peano 
arithmetic is undecidable. 

Tarski showed that the theory of the real closed fields (inequalities of polyno- 
mial expressions) admits quantifier elimination, [16] however his algorithm had 
impractical (non-elementary) complexity. Later, the cylindrical algebraic de- 
composition (CAD) 2, Ch. 11] method was introduced, with doubly exponential 
complexity, which is unavoidable in the worst case [2J §11.4]. Our experiments 
with both Mathematica and Qepcad, both of which implement CAD, as well 
as with Reduce/Redlog, which implement various algorithms for quantifier 
elimination, showed us that combinatorial blowup occurs very quickly. For such 
techniques to be interesting in practice, practical complexity should be lowered. 
Perhaps our technique could help. There are, however, significant difficulties in 
that respect. Our technique starts with some single model of the target formula 
over the rational numbers; but a system of nonlinear inequalities needs not have 
rational models when it is not full-dimensional (for instance, X 2 — 2). Our tech- 

1: http : //lira.gf orge . avacs . org/toolpaper/randPrsb .hs 

12 We used the command line randprsb 7 -10 10 n i where n is the depth parameter 
(here, 14, 15 or 16) and i ranges in [0,99]. 



13 



nique reduces the geometrical computations to computations on conjunctions; 
but in the nonlinear case, single inequalities can be reduced to disjunctions. As 
an example, X 2 > 4 is reduced to X < —2 V X > 2. Most importantly, our 
technique relies at several steps on the availability of a decision procedure that 
stays efficient even when the answer is negative. 

Regarding the mixed integer / real problems, the Lira tool implements 
quantifier elimination using a weak form of Biichi automata matching the 6-ary 
expression of the integers or reals, where b is an arbitrary base. [3] The output of 
the process is an automaton and not a readable formula. While it is possible to 
decide a closed formula, and to obtain one model from a satisfiable non-closed 
formula, it is an open problem how to efficiently reconstruct a quantifier-free 
formula from the resulting automaton. The automaton construct is unsuitable 
for large coefficients (as our examples obtained from the analysis of floating- 
point programs). Even on examples with small coefficients, the tool was unable 
to complete quantifier elimination without blowing up. We think therefore that 
it would be interesting to be able to apply our technique to the mixed integer 
/ real problems, but there are difficulties: the algorithms on integer polyhedra 
are considerably more complex than on rational polyhedra. 

A classical objection to automatic program analysis tools meant to prove 
the absence of bugs is that these tools could themselves contain bugs. Our 
method uses complex algorithms (SMT-solving, polyhedron projection) as sub- 
procedures. We consider developing techniques so that the algorithm outputs 
easily-checkable proofs or "proof witnesses" of the correctness of its computation. 
Furthermore, we showed in earlier publications [12 that certain program analy- 
sis tasks were equivalent to quantifier elimination problems; that is, an effective 
static analyzer can be extracted from the quantifier-free form of an analyzer 
specification. This therefore suggests a new way for writing safe static analyz- 
ers: instead of painstakingly writing an analyzer, then proofs of correctness in a 
proof assistant [15j . one could formulate the analysis problem as an equivalent 
quantifier elimination problem, with a relatively simple proof of equivalence, 
then apply a "certified" quantifier elimination procedure in order to extract the 
effective analyzer. 
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